Skip to content

fix security scan false positives for web-security skills#18

Merged
GangGreenTemperTatum merged 1 commit into
mainfrom
ads/cap-976-investigate-the-failing-capabilities-github-actions-workflow
May 22, 2026
Merged

fix security scan false positives for web-security skills#18
GangGreenTemperTatum merged 1 commit into
mainfrom
ads/cap-976-investigate-the-failing-capabilities-github-actions-workflow

Conversation

@GangGreenTemperTatum
Copy link
Copy Markdown
Contributor

This change fixes a defensible false-positive path in the capabilities security scan workflow.

The failing GitHub Actions run was the Security Scan workflow on main after a runtime-only change to the web-security capability. That change added waymore, but the workflow rescanned the entire capability root and failed on existing high and critical findings in offensive-security instructional content. Two classes of false positives were involved.

First, the workflow scanned the whole capability directory with --recursive --lenient. In lenient mode, the scanner falls back to treating plain markdown folders as pseudo-skills when SKILL.md is absent. That caused exploit-verifier/references/checklists and exploit-verifier/references/ai-payloads to be scanned as if they were skills, and their prompt-injection examples tripped the high-severity gate.

Second, a small number of findings were policy mismatches for this repository’s intended content. csp-bypass was flagged by BEHAVIOR_BASH_TAINT_FLOW because the scanner interpreted the literal string unsafe-eval inside a grep command as though the shell snippet were executing eval. blind-ssrf-chains was flagged by YARA_command_injection_generic because it intentionally documents a literal exploit payload string inside an offensive-security skill.

The fix is narrow. The workflow now scans capabilities/<cap>/skills instead of the capability root, which limits scanning to actual skills and avoids pulling agents and references directories into fallback skill discovery. It also removes --lenient from the CI invocation so reference markdown is not treated as malformed skills. In the repository scan policy, BEHAVIOR_BASH_TAINT_FLOW and YARA_command_injection_generic are demoted to MEDIUM with repository-specific rationale tied to instructional shell snippets and exploit payload examples in security skills.

Validation was done locally by reproducing the failing scanner invocation, then rerunning it with the updated workflow target and policy. The web-security skill scan now passes the high failure threshold with zero critical and zero high findings. I also ran pre-commit run --files .github/workflows/security-scan.yml scan-policy.yaml, which passed, and just validate, which passed with the repo’s existing local-tool warnings for optional environment checks.

@GangGreenTemperTatum GangGreenTemperTatum marked this pull request as ready for review May 22, 2026 13:48
@GangGreenTemperTatum GangGreenTemperTatum merged commit dd59c19 into main May 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant